Putting a label on someone: impact of schizophrenia stigma on emotional mimicry, liking, and interpersonal closeness.

Affiliation is both an antecedent and a consequence of emotional mimicry (i.e. imitating a counterpart's emotional expression). Thus, interacting with a disliked partner can decrease emotional mimicry, which in turn can further decrease liking. This perpetuating circle has not been investigated in the context of mental health stigma yet. The present study tested the influence of the label "schizophrenia" on liking, interpersonal closeness, and emotional mimicry. In an online experiment (n = 201), participants recruited from the general population saw several videos of actors displaying emotional expressions. Actors were described with one of four labels: "schizophrenia", "healthy", "diabetes", and a negative adjective (e.g. "hot-tempered"). Emotional mimicry was measured using OpenFace 2.2. Liking and interpersonal closeness were assessed with questionnaires. Overall, compared to other labels, participants reported less liking and interpersonal closeness to the actor with the schizophrenia label. However, no effect on emotional mimicry was found. The decreased liking of the schizophrenia actors was explained by a lack of knowledge about schizophrenia and the explicit stigma of schizophrenia. Our study contributes to the literature by highlighting the need to reduce the stigma of schizophrenia.

Impaired perception of a partner's synchronizing behavior reduces positive attitude toward humanoid robot in schizophrenia patients.

As interpersonal synchrony plays a key role in building rapport, the perception of another agent's synchronizing behavior could be an important feature to assess, especially with patients with social deficits such as in schizophrenia. Twenty-four schizophrenia patients and twenty-four matched healthy controls performed jointly fitness movements with another agent embodied by a humanoid robot which was programmed to either synchronize with the participants or move at a fixed frequency with them. Self-report of participants' perception of the robot's synchronizing behavior was collected after each interaction. Results indicated that patients were impaired in their ability to accurately perceive the robot's synchronizing behavior. Patients' subjective perception of the robot's synchronizing behavior was associated with positive attitude toward it, suggesting that the belief to be synchronized with others could have similar impact on affiliation than real interpersonal synchrony. It leads to new perspectives for understanding social deficits in people with severe mental illness.

Gesture-speech synchrony in schizophrenia: A pilot study using a kinematic-acoustic analysis.

Severe impairment of social functioning is the core feature of schizophrenia that persists despite treatment, and contributes to chronic functional disability. Abnormal non-verbal behaviors have been reported during interpersonal interactions but the temporal coordination of co-speech gestures with language abilities have been poorly studied to date in this pathology. Using the dynamical systems framework, the goal of the current study was to investigate whether gestures and speech synchrony is impaired in schizophrenia, exploring a new approach to report communicational skill disorders. Performing the first continuous kinematic-acoustic analysis in individuals with schizophrenia, we examined gesture-speech synchrony in solo spontaneous speech and in sensorimotor synchronization task. The experimental group consisted of twenty-eight participants with a diagnosis of schizophrenia and the control group consisted of twenty-four healthy participants age-gender-education matched. The results showed that spontaneous gesture-speech synchrony was preserved while intentional finger tapping-speech synchrony was impaired. In sensorimotor synchronization task, the schizophrenia group displayed greater asynchronies between finger tapping and syllable uttering and lower stability of coordination patterns. These findings suggest a specific deficit in time delay of information circulation and processing, especially in explicit functions. Thus, investigating intrapersonal coordination in schizophrenia may constitute a promising window into brain/behavior dynamic relationship.

Human unintentional and intentional interpersonal coordination in interaction with a humanoid robot.

In order to establish natural social synchrony between two humans, two requirements need to be fulfilled. First, the coupling must be bi-directional. The two humans react to each other's actions. Second, natural social bodily synchronization has to be intentional or unintentional. Assuming that these essential aspects of human-human interactions are present, the present paper investigates whether similar bodily synchrony emerges between an interacting human and an artificial agent such as a robot. More precisely, we investigate whether the same human unintentional rhythmic entrainment and synchronization is present in Human Robot Interaction (HRI). We also evaluate which model (e.g., an adaptive vs non adaptive robot) better reproduces such unintentional entrainment. And finally, we compare interagent coordination stability of the HRI under 1) unidirectional (robot with fixed frequency) versus bidirectional (robot with adaptive frequency) rhythmic entrainment and 2) human intentional versus unintentional coupling. Fifteen young adults made vertical arm movements in front of the NAO robot under five different conditions of intentional/unintentional and unidirectional/bidirectional interactions. Consistent with prior research investigating human-human interpersonal coordination, when humans interact with our robot, (i) unintentional entrainment was present, (ii) bi-directional coupling produced more stable in-phase un-intentional and intentional coordination, (iii) and intentional coordination was more stable than unintentional coordination. To conclude, this study provides a foundation for modeling future social robots involving unintentional and bidirectional synchronization-aspects which seem to enhance humans' willingness to interact with robots.

Effects of unintentional coordination on attentional load.

The goal of this study was to evaluate the impact of unintentional (spontaneous) coordination on high attentional visual load. More precisely, we wondered whether such coordination could free up some attentional resources and help improve performance in other more demanding attentional tasks. An experiment was performed in which participant attentional allocation was challenged by performing three tasks simultaneously while simultaneously being induced to unintentional entrain to an environmental rhythm. The first task was an interception task associated with a Stroop test to increase their attentional load. The second task was a reaction time test to alarms in different modalities (auditory, visual and bimodal) which was used to assess participant attentional load. The third task was a motor task in which participants were asked to swing their legs at a preferred frequency. The interface background brightness intensity was either synchronized in real time using a bidirectional coupling to participant leg movement or the background brightness was not changing at all. Our results on the reaction time task demonstrated that participants exhibited better reaction times for alarms in the bimodal condition than in the auditory condition and lastly for the visual condition. Also, participants exhibited a lower reaction time to alarms when the background brightness was synchronizing with their leg regardless the alarm modality. Overall, our study suggests a beneficial effect of unintentional environmental coordination on attentional resource allocation and highlights the importance of bidirectionality in interaction.

Toward an Emotional Individual Motor Signature.

Bodily expression of felt emotion has been documented in the literature. However, it is often associated with high motor variability between individuals. This study aimed to identify individual motor signature (IMS) of emotions. IMS is a new method of motion analysis and visualization able to capture the subtle differences in the way each of us moves, seen as a kinematic fingerprint. We hypothesized that the individual motor signature would be different depending on the induced emotional state and that an emotional motor signature of joy and sadness common to all participants would emerge. For that purpose, we elicited these emotions (joy, sadness, and a neutral control emotion) in 26 individuals using an autobiographical memory paradigm, before they performed a motor improvization task (e.g., the mirror game). We extracted the individual motor signature under each emotional condition. Participants completed a self-report emotion before and after each trial. Comparing the similarity indexes of intra- and inter-emotional condition signatures, we confirmed our hypothesis and showed the existence of a specific motor signature for joy and sadness, allowing us to introduce the notion of emotional individual motor signature (EIMS). Our study indicates that EIMS can reinforce emotion discrimination and constitutes the first step in modeling emotional behavior during individual task performances or social interactions.

Decoding identity from motion: how motor similarities colour our perception of self and others.

For more than 4 decades, it has been shown that humans are particularly sensitive to biological motion and extract socially relevant information from it such as gender, intentions, emotions or a person's identity. A growing number of findings, however, indicate that identity perception is not always highly accurate, especially due to large inter-individual differences and a fuzzy self-recognition advantage compared to the recognition of others. Here, we investigated the self-other identification performance and sought to relate this performance to the metric properties of perceptual/physical representations of individual motor signatures. We show that identity perception ability varies substantially across individuals and is associated to the perceptual/physical motor similarities between self and other stimuli. Specifically, we found that the perceptual representations of postural signatures are veridical in the sense that closely reflects the physical postural trajectories and those similarities between people' actions elicit numerous misattributions. While, on average, people can well recognize their self-generated actions, they more frequently attribute to themselves the actions of those acting in a similar way. These findings are consistent with the common coding theory and support that perception and action are tightly linked and may modulate each other by virtue of similarity.

Influence of perceived emotion and gender on social motor coordination.

Theorists have long postulated that facial properties such as emotion and sex are potent social stimuli that influence how individuals act. Yet extant scientific findings were mainly derived from investigations on the prompt motor response upon the presentation of affective stimuli, which were mostly delivered by means of pictures, videos, or text. A theoretical question remains unaddressed concerning how the perception of emotion and sex would modulate the dynamics of a continuous coordinated behaviour. Conceived in the framework of dynamical approach to interpersonal motor coordination, the present study aimed to address this question by adopting the coupled-oscillators paradigm. Twenty-one participants performed in-phase and anti-phase coordination with two avatars (male and female) displaying three emotional expressions (neutral, happy, and angry) at different frequencies (100% and 150% of the participant's preferred frequency) by executing horizontal rhythmic left-right oscillatory movements. Time to initiate movement (TIM), mean relative phase error (MnRP), and standard deviation of relative phase (SDRP) were calculated as indices of reaction time, deviation from the intended pattern of coordination, and coordination stability, respectively. Results showed that in anti-phase condition at 150% frequency, MnRP was lower with the angry and the female avatar. In addition, coordination was found to be more stable with the male avatar than the female one when both displaying neutral emotion. But the happy female avatar was found to elicit more stable coordination than the neutral female avatar. These results implied that individuals are more relaxed to coordinate with the female than the male, and the sensorimotor system becomes more flexible to coordinate with an angry person. It is also suggested social roles influence how people coordinate, and individuals attend more to interact with a happy female. In sum, the present study evidenced that social perception is embodied in the interactive behaviour during social interaction.

Towards an Embodied Signature of Improvisation Skills.

Improvisation is not limited to the performing arts, but is extended to everyday life situations such as conversations and decision-making. Due to their ubiquitous nature, improvisation skills have received increasing attention from researchers over the last decade. A core challenge is to grasp the complex creative processes involved in improvisation performance. To date, many studies have attempted to provide insight on brain activity and perceptual experiences when perceiving a performance, especially in musical or artistic form. However, watching/listening a performance is quite different than acting in a performance or performing daily-life activities. In this Perspective, we discuss how researchers have often missed key points concerning the study of improvisation skills, especially by ignoring the central role of bodily experiences in their formation. Furthermore, we consider how the study of (neglected) motor component of improvisation performance can provide valuable insights into the underlying nature of creative processes involved in improvisation skills and their acquisition. Finally, we propose a roadmap for studying improvisation from the acquisition of kinematic data in an ecological context to analysis, including the consideration of the coalition of (individual, environmental and task) constraints in the emergence of improvised behaviors.

Using mimicry of body movements by a virtual agent to increase synchronization behavior and rapport in individuals with schizophrenia.

Synchronization of behavior such as gestures or postures is assumed to serve crucial functions in social interaction but has been poorly studied to date in schizophrenia. Using a virtual collaborative environment (VCS), we tested 1) whether synchronization of behavior, i.e., the spontaneous initiation of gestures that are congruent with those of an interaction partner, was impaired in individuals with schizophrenia compared with healthy participants; 2) whether mimicry of the patients' body movements by the virtual interaction partner was associated with increased behavioral synchronization and rapport. 19 patients and 19 matched controls interacted with a virtual agent who either mimicked their head and torso movements with a delay varying randomly between 0.5 s and 4 s or did not mimic, and rated feelings of rapport toward the virtual agent after each condition. Both groups exhibited a higher and similar synchronization behavior of the virtual agent forearm movements when they were in the Mimicry condition rather than in the No-mimicry condition. In addition, both groups felt more comfortable with a mimicking virtual agent rather than a virtual agent not mimicking them suggesting that mimicry is able to increase rapport in individuals with schizophrenia. Our results suggest that schizophrenia cannot be considered anymore as a disorder of imitation, particularly as regards behavioral synchronization processes in social interaction contexts.

Standing or swaying to the beat: Discrete auditory rhythms entrain stance and promote postural coordination stability.

Humans seem to take social and behavioral advantages of entraining themselves with discrete auditory rhythms (e.g., dancing, communicating). We investigated the benefits of such an entrainment on posture during standing (spontaneous entrainment) and during a whole-body swaying task (intentional synchronization). We first evaluated how body sway was entrained by different auditory metronome frequencies (0.25, 0.5, and 1.0Hz). We then assessed the stabilizing role of auditory rhythms on postural control, characterized in a dynamical systems perspective by informational anchoring of the head (local stabilization) and fewer transitions from in-phase to anti-phase ankle-hip coordination (global stabilization). Our results revealed in both situations an entrainment of postural movements by external rhythms. This entrainment tended to be more effective when the metronome frequency (0.25Hz) was close to the dominant sway frequency. Particularly, we found during intentional synchronization that head movements were less variable when paced by a slower beat (informational anchoring), and that phase transitions between the two stable patterns in postural dynamics were delayed. Our findings demonstrate that human bipedal posture can be actively or spontaneously modulated by an external discrete auditory rhythm, which might be exploited for the purpose of learning and rehabilitation.

Influence of facial feedback during a cooperative human-robot task in schizophrenia.

Rapid progress in the area of humanoid robots offers tremendous possibilities for investigating and improving social competences in people with social deficits, but remains yet unexplored in schizophrenia. In this study, we examined the influence of social feedbacks elicited by a humanoid robot on motor coordination during a human-robot interaction. Twenty-two schizophrenia patients and twenty-two matched healthy controls underwent a collaborative motor synchrony task with the iCub humanoid robot. Results revealed that positive social feedback had a facilitatory effect on motor coordination in the control participants compared to non-social positive feedback. This facilitatory effect was not present in schizophrenia patients, whose social-motor coordination was similarly impaired in social and non-social feedback conditions. Furthermore, patients' cognitive flexibility impairment and antipsychotic dosing were negatively correlated with patients' ability to synchronize hand movements with iCub. Overall, our findings reveal that patients have marked difficulties to exploit facial social cues elicited by a humanoid robot to modulate their motor coordination during human-robot interaction, partly accounted for by cognitive deficits and medication. This study opens new perspectives for comprehension of social deficits in this mental disorder.

Likability's Effect on Interpersonal Motor Coordination: Exploring Natural Gaze Direction.

Although existing studies indicate a positive effect of interpersonal motor coordination (IMC) on likability, no consensus has been reached as for the effect of likability back onto IMC. The present study specifically investigated the causal effect of likability on IMC and explored, by tracking the natural gaze direction, the possible underlying mechanisms. Twenty-two participants were engaged in an interpersonal finger-tapping task with a confederate in three likability conditions (baseline, likable, and unlikable), while wearing an eye tracker. They had to perform finger tapping at their comfort tempo with the confederate who tapped at the same or 1.5 times of the participant's preferred frequency. Results showed that when tapping at the same frequency, the effect of likability on IMC varied with time. Participants coordinated at a higher level in the baseline condition at the beginning of the coordination task, and a facilitative effect of likability on IMC was revealed in the last session. As a novelty, our results evidenced a positive correlation between IMC and the amount of gaze onto the coordination partner's movement only in the likable condition. No effect of likability was found when the confederate was tapping at 1.5 times of the participant's preferred frequency. Our research suggests that the psychosocial property of the coordinating partner should be taken into consideration when investigating the performance of IMC and that IMC is a parameter that is sensitive to multiple factors.

Understanding the Impact of Expertise in Joint and Solo-Improvisation.

Joint-improvisation is not only an open-ended creative action that two or more people perform together in the context of an artistic performance (e.g., theatre, music or dance). Joint-improvisation also takes place in daily life activities when humans take part in collective performance such as toddlers at play or adults engaged in a conversation. In the context of this article, joint-improvisation has been looked at from a social motor coordination perspective. In the literature, the nature of the social motor coordination characteristics of joint-improvisation for either the creative aspect or daily life features of this motor performance remains unclear. Additionally, both solo-improvisation and joint-improvisation need to be studied conjointly to establish the influence of the social element of improvisation in the emergence of multi-agent motor coordination. In order to better understand those two types of improvisation, we compared three level of expertise - novice, intermediate and professional in dance improvisation to identify movement characteristics for each of the groups. Pairs of the same level were asked to improvise together. Each individual was also asked to perform an improvisation on his/her own. We found that each of the three groups present specific movement organization with movement complexity increasing with the level of expertise. Experts performed shorter movement duration in conjunction with an increase range of movement. The direct comparison of individual and paired Conditions highlighted that the joint-improvisation reduced the complexity of the movement organization and those for all three levels while maintaining the differences between the groups. This direct comparison amongst those three distinct groups provides an original insight onto the nature of movement patterns in joint-improvisation situation. Overall, it reveals the role of both individual and collective properties in the emergence of social coordination.

Unravelling socio-motor biomarkers in schizophrenia.

We present novel, low-cost and non-invasive potential diagnostic biomarkers of schizophrenia. They are based on the 'mirror-game', a coordination task in which two partners are asked to mimic each other's hand movements. In particular, we use the patient's solo movement, recorded in the absence of a partner, and motion recorded during interaction with an artificial agent, a computer avatar or a humanoid robot. In order to discriminate between the patients and controls, we employ statistical learning techniques, which we apply to nonverbal synchrony and neuromotor features derived from the participants' movement data. The proposed classifier has 93% accuracy and 100% specificity. Our results provide evidence that statistical learning techniques, nonverbal movement coordination and neuromotor characteristics could form the foundation of decision support tools aiding clinicians in cases of diagnostic uncertainty.

Social-motor experience and perception-action learning bring efficiency to machines.

Lake et al. proposed a way to build machines that learn as fast as people do. This can be possible only if machines follow the human processes: the perception-action loop. People perceive and act to understand new objects or to promote specific behavior to their partners. In return, the object/person provides information that induces another reaction, and so on.

Postural Coordination during Socio-motor Improvisation.

Human interaction often relies on socio-motor improvisation. Creating unprepared movements during social interaction is not a random process but relies on rules of synchronization. These situations do not only involve people to be coordinated, but also require the adjustment of their posture in order to maintain balance and support movements. The present study investigated posture in such a context. More precisely, we first evaluated the impact of amplitude and complexity of arm movements on posture in solo situation. Then, we assessed the impact of interpersonal coordination on posture using the mirror game in which dyads performed improvised and synchronized movements (i.e., duo situation). Posture was measured through ankle-hip coordination in medio-lateral and antero-posterior directions (ML and AP respectively). Our results revealed the spontaneous emergence of in-phase pattern in ML direction and antiphase pattern in AP direction for solo and duo situations. These two patterns respectively refer to the simultaneous flexion/extension of the ankles and the hips in the same or opposite direction. It suggests different functional roles of postural coordination patterns in each direction, with in-phase supporting task performance in ML (dynamical stability) and antiphase supporting postural control in AP (mechanical stability). Although amplitude of movement did not influence posture, movement complexity disturbed postural stability in both directions. Conversely, interpersonal coordination promoted postural stability in ML but not in AP direction. These results are discussed in terms of the difference in coupling strength between ankle-hip coordination and interpersonal coordination.

Humanoid robots versus humans: How is emotional valence of facial expressions recognized by individuals with schizophrenia? An exploratory study.

BACKGROUND: The use of humanoid robots to play a therapeutic role in helping individuals with social disorders such as autism is a newly emerging field, but remains unexplored in schizophrenia. As the ability for robots to convey emotion appear of fundamental importance for human-robot interactions, we aimed to evaluate how schizophrenia patients recognize positive and negative facial emotions displayed by a humanoid robot. METHODS: We included 21 schizophrenia outpatients and 17 healthy participants. In a reaction time task, they were shown photographs of human faces and of a humanoid robot (iCub) expressing either positive or negative emotions, as well as a non-social stimulus. Patients' symptomatology, mind perception, reaction time and number of correct answers were evaluated. RESULTS: Results indicated that patients and controls recognized better and faster the emotional valence of facial expressions expressed by humans than by the robot. Participants were faster when responding to positive compared to negative human faces and inversely were faster for negative compared to positive robot faces. Importantly, participants performed worse when they perceived iCub as being capable of experiencing things (experience subscale of the mind perception questionnaire). In schizophrenia patients, negative correlations emerged between negative symptoms and both robot's and human's negative face accuracy. CONCLUSIONS: Individuals do not respond similarly to human facial emotion and to non-anthropomorphic emotional signals. Humanoid robots have the potential to convey emotions to patients with schizophrenia, but their appearance seems of major importance for human-robot interactions.

Dynamic similarity promotes interpersonal coordination in joint action.

Human movement has been studied for decades, and dynamic laws of motion that are common to all humans have been derived. Yet, every individual moves differently from everyone else (faster/slower, harder/smoother, etc.). We propose here an index of such variability, namely an individual motor signature (IMS) able to capture the subtle differences in the way each of us moves. We show that the IMS of a person is time-invariant and that it significantly differs from those of other individuals. This allows us to quantify the dynamic similarity, a measure of rapport between dynamics of different individuals' movements, and demonstrate that it facilitates coordination during interaction. We use our measure to confirm a key prediction of the theory of similarity that coordination between two individuals performing a joint-action task is higher if their motions share similar dynamic features. Furthermore, we use a virtual avatar driven by an interactive cognitive architecture based on feedback control theory to explore the effects of different kinematic features of the avatar motion on coordination with human players.